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ABSTRACT 



The ways in which experience and academic training influence 
the types of questions therapists ask of their clients were studied. 
Psychologists and psychology students of varying degrees of expertise (n=138) 
were provided an artificial intelligence program that simulated a date rape 
client's responses to typed questions and were asked to conduct a "clinical 
interview" by typing in their questions and other responses. Clinical t 

experience was found to be unrelated to participants' levels of confidence, 
perceived expertise, or efficiency in generating or analyzing questions, but 
both experience and level of training did affect the type of responses and 
questions of participants. The significance of these findings is discussed. 
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Psychologists and psychology students (n = 138) were provided an artificial-intelligence program 
that simulated a date rape client’s responses to typed questions and asked to conduct a “clinical 
interview” by typing in their questions and other responses. Clinical experience was found to be 
unrelated to participants’ levels of confidence, perceived expertise, or efficiency in generating or 
analyzing questions. But both experience and level of training did affect the type of responses and 
questions of participants. The significance of these findings is discussed. 




3 



Expertise in Psychological Inquiry 

3 



Expertise in Psychological Inquiry 

During the past two decades, psychologists have been responsible for a considerable and 
growing literature on the nature of expertise and how it develops (Bloom, 1985; Cooke, 1992; 
Ericsson & Crutcher, 1990; Ericsson & Charness 1994; Ericsson, Krampe, & Tesch-Romer, 

1993; Hayes, 1981; Patel & Groen, 1988). Most such studies have focused on fields outside of 
psychology. They have addressed expertise in populations such as college students 
(Radhakrishnan, Arrow, & Sniezek, 1996), factory workers (Garmer, Dahlman, & Sperling, 
1995), entrepreneurs (Mitchell & Chesteen, 1995), and classroom teachers (Pugach, & Johnson, 
1995). The domains have included such areas as playing chess, planning military strategy, reading 
x-rays, and programming computers. 

The purpose of this paper was to add to the still-small literature concerning expertise in 
the domain of professional psychology. Specifically, this study explores the ways experience and 
academic training influence the types of questions therapists ask of their clients. Though a number 
of other factors may prove more important to therapeutic outcomes, therapists’ modes of inquiry 
were examined in this study because this task appears to be foundational to other therapeutic 
activities. 

The literature on critical judgment in psychology and other fields emphasizes the 
importance of pattern heuristics, forward reasoning, and procedural knowledge. In most fields, 
the literature demonstrates a connection between these components with experience and training. 
In psychology, however, the connection appears tenuous at best. 
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the literature demonstrates a connection between these components with experience and training. 
In psychology, however, the connection appears tenuous at best. 

Psychologists interested in the nature of expertise have focused on how experts across 
different fields demonstrate similarities in their behaviors (Bloom, 1985; Cooke, 1992; de Groot, 
1978; Ericsson & Crutcher, 1990, Ericsson & Chamess 1994; Ericsson, Krampe, & Tesch- 
Romer, 1993; Hayes, 1981; Patel & Groen, 1988). In the 1970s, researchers such as Chase and 
Simon (1973) believed experts in a single field established basic skills that would allow them to 
quickly master other fields. Many of these studies relied on the work of de Groot (1946, 1978) 
and his seminal study on expert chess players, showing that expert chess players were better able 
than novices to remember patterns of chess sequences. One’s knowledge of the domain was 
secondary to the knowledge of the heuristic. As many have noted (eg., Ericsson & Chamess, 
1994; Rosenbloom & Newell, 1986), these arguments were short lived, but they were the 
foundation of the psychology of expertise. 

Pattern Heuristics 

As more theorists rejected the plausibility of any single heuristic to provide expertise in 
multiple fields, researchers turned to subdisciplines. Theories argued that expert chess players 
created specialized heuristics to assist them in their field of study. The skills used in chess were 
different that the skills required to master, say, physics, but the heuristics for mastering chess were 
identifiable. Experts were viewed as people who had mastered advanced methods of solving 
problems within a specific domain (Schneider, Gruber, Gold, & Opwis, 1993). They grasped more 
information through perceiving patterns, rather than focusing on the individual facets of the 
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problem. This decreased the time required to reach a solution and produced more accurate 
solutions (Cooke, 1992; Dreyfus & Dreyfus, 1986). The specialized heuristics also assist the 
expert in conceptualizing the problem structure (Popal, 1982). Rather than organize knowledge 
based on superficial and often irrelevant as novices did (Etringer, Hillerbrand, & Claibom, 1995). 

The development of pattern heuristics corresponds with a shift in the problem solver’s 
methodology. As they gain expertise and begin to predict facets within routine problems, problem 
solvers learn to work forward from the facts to a desired outcome (Hunt, 1989). They work from 
a given pattern of information toward an expected conclusion. Novices work backward from a 
specific goal or a theoretical conceptualization of the client to the facts (Patel & Groen, 1991). 
They formulate tentative solutions and seek confirmatory or disconfirmatory evidence for the 
solution. Patel and Groen (1986) found that all of their medical subjects who made completely 
accurate diagnoses used “pure” forward reasoning, but that none of the subjects who made 
inaccurate diagnoses utilized this technique. 

Breaking the Rules. 

What happens when a pattern does not appear to develop or when the developing pattern 
is unfamiliar? Goldin (1978) noted that although experts in chess are able to recognize familiar 
positions and patterns quickly, they are considerably less efficient in recognizing atypical patterns. 
Patel and Groen (1991) found that expert medical clinicians faired better than novices only when 
working on a normally-structured problem. When the structure of the problem was changed, 
novices continued their strategy of reasoning without significant change in their level of 
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performance. The performance of the more experienced physicians, on the other hand, 
deteriorated to the level of the novices’ performance. 

The same changes were noted by Hillerbrand and Claibom (1990) concerning 
psychological judgment and diagnosis. When diagnostic patterns became less consistent with 
realistic diagnostic patterns, the participants’ cognitive processes became less efficient. This 
finding was especially noteworthy for experts, who made rapid and brad inferences, suitable 
predictions but engaged in less hypothesis testing than novices when examining a well-structured 
problem. Decreased structure appears to affect the experts’ ability to automate their reasoning 
processes. 

Literature on Psychological Expertise 

Little research exists concerning the process of making a psychological diagnosis 
(Hillerbrand & Claibom, 1990). It is likely that some of the same skills found in expert 
psychotherapists, such as active listening (Vargo & McFarlane, 1994) and attending to client 
statements (Hayes & Wolf, 1984) and nonverbal behavior (Clinard, 1985), would also exist for 
expert diagnosticians. However, there is no literature examining such hypotheses. 

As Goodyear (in press) notes, experience is often used as a proxy for expertise. Ericsson 
and Lehmann’s (1996) argument that 10 years are required for the development of expertise in 
any field continues to champion the correlation of experience with expertise. The therapist, 
however, must be deliberately attempting to develop skills for experience to have the desired 
effect on expertise (Clark, 1992; Ericsson et al., 1993). 
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Dawes’ (1989, 1994) contention that practice and experience alone appear insufficient to 
produce expertise in psychological inquiry is well supported. Garb’s (1989) review of the 
literature is a good example. One possible way of overcoming the limitations of effectiveness 
would be to engage in specialized, deliberate practice and maintain a broad, cognitively complex 
perspective on encountered problems. For example, the literature on rape victims suggests that 
these individuals had a higher than average rate of prior sexual abuse and a higher than average 
rate of attempted suicides (Ben & Silfen, 1993). Women who were molested, raped, or 
experienced attempted rape were also more likely than average to tax their capacity for handling 
depression and anxiety (Kilpatrick, 1985). Because experience lends itself to predicting commonly 
encountered patterns, expert therapists who had encountered rape victims would be more likely to 
identify issues of suicidal ideation, anxiety, and depression. The connection between specialized 
experience and effective judgments has not been adequately addressed in the literature, though it 
has been discussed (Beutler, 1989). 

Level of Academic Training 

Little research exists on the processes psychologists go through when making their 
assessment or diagnosis (Gelso & Fretz, 1992) or on the role of academic training on the 
assessment process. Early studies in assessment focused on the relationship between diagnosis and 
the adherence to a single theoretical perspective (McArthur, 1954; Koester, 1954). The argument 
was made that theoretical orientations should be rigidly adhered to for effective diagnoses. More 
recently, Lanyon and Goodstein’s (1982) review found that effective diagnosticians were more 
likely to seek client feedback concerning the accuracy of the proposed diagnosis. However, these 
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are preliminary studies, and they do not address the relationship between the assessment process 
and academic training. 

Most research on the relationship between training and the diagnostic process focused on 
the ability of experts to form accurate diagnoses (e.g., Christensen & Jacobsen, 1994; Dawes, 
1994; and Goldberg, 1959, 1968; Hattie, Sharpley, & Rogers, 1984, and Stein & Lambert 1989, 
1995). This contention has received support through Smith and Glass’ meta-analysis on the 
effectiveness of psychotherapy which found that credentials (e g., Ph D., M.D., no advanced 
degree, etc.) made no statistically significant impact. At best, the current body of literature 
supports the contention that professionals have roughly the same client outcomes as 
paraprofessionals (Berman & Norton, 1985). 

The inability of trained psychologists to out perform minimally trained non-psychologists 
in diagnosis and treatment also appears to extend to non-diagnostic judgments. Interviews, the 
primary source of therapeutic information, are now considered unreliable (Dawes, 1994, Gelso, 
1992). However, it is possible that academic training has an impact on the types of inquiries made 
in ways that are not directly related to diagnostic accuracy. For example, it has long been argued 
that gathering a life history is an important assessment task (Gelso, 1992). Do therapists with 
doctoral degrees emphasize these issues more the psychologists in training? Does training 
encourage psychologists to ask the same types of questions, or does it encourage them to 
creatively explore individualistic approaches? 

Q’Byme’s (1993) study provided an initial investigation into these processes. She found 
that expert psychologists tended to request significantly more client information, asked fewer 
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crisis-related questions, and asked more questions on a given topic. However, some of these 
findings, such as the tendency to ask more questions, contradict what would be expected from the 
general literature one expertise. Further investigations are warranted. 

Literature on Threats to Psychological Expertise 

A significant difficulty with diagnostic criteria involves the ability of therapists to develop 
accurate and reliable assessments of client problems. Despite the apparent weaknesses of training 
and experience to produce reliable diagnosticians, therapists report high levels of confidence in 
their judgments. Vallone, Griffin, Lin, and Ross (1990) reported that therapists’ levels of 
confidence are inversely correlated with accuracy (see also Arnold, 1993). As psychologists 
become less accurate, they evoke high manifest levels of confidence (Hillerbrand & Claibom, 
1990). There is a long history of literature indicating that therapists will report higher levels of 
confidence than they merit (Dumont, 1993; Dunning, Griffin, Milojkovic, & Ross, 1990; 

Glidewell & Livert, 1992; and Tracy, 1991). 

Despite their problem with overconfidence, experienced therapists are better at describing 
which of their judgments are more likely to be accurate (Garb, 1986, 1989). Others have noted 
that confidence changes with information. Oskamp (1982) found that clinicians confidence in their 
judgments grow with additional knowledge of the client, and Hillerbrand and Claiborn (1990) 
noted that experts felt more knowledgeable and confident in their judgments even though their 
cognitive processing did not statistically differ from those of novices. 

One reason for the presence of overconfidence is the tendency for experts to automate 
their procedures. Less information is required to make these judgments because patterns are 
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quickly identified (Dumont, 1991). However, when experts confront variants to typically 

encountered problems, the investigative process may be arrested prematurely. These are the cases 

where experts do not appear to perform better than novices (Patel & Groen, 1991). 

In this study four research questions were addressed: 

Question 1 . As clinical experience increases, do psychologists use forward reasoning to change 
their mode of inquiry? 

Question 2 . Does level of academic training affect the amount and type of information 
psychologists seek? 

Question 3 , As psychologists gain experience, does their level of confidence rise? 

Question 4, As experience increases, do psychologists take less time to formulate or analyze their 
questions? 

Method 

Participants 

Participants were 138 (86 men; 52 women) psychologists or psychology students. Most 
(n = 68; 49%) had a doctorate in psychology or had been advanced to candidacy in a doctoral 
program (n = 14, 10%). The rest (n = 56; 41%) ranged from those who were just beginning 
graduate training to doctoral candidates. 

Most (n = 106; 77%) participants were White. The rest described themselves as Asian (n = 
6; 4%), Black (n = 4; 3%), Latino (n = 2; 1%), or “other” (n = 20; 15%). Participants’ experience 
ranged from 0 to 3 1 years (M = 6.03, SD = 7. 19). The largest group (n = 50; 36%) described 
themselves as academic faculty. The rest were clinicians at Veteran’s Administration hospitals 
(n = 18; 13%), community mental health center (n = 19; 14%), university counseling centers 
(n = 21; 15%). some (n = 30; 22%) were not presently employed. . 
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Materials 

An interactive, artificially intelligent, computer program, modeled after O’Byme’s (1993) 
study, was written for this study. The program approximates a real-life assessment situation and 
simulates a client’s responses to questions. The “client’s” presenting problem involved a recent 
date rape experience. Participants were informed that she had been referred by a colleague who 
diagnosed her with PTSD. 

Participants could ask any question they wished within the confines of 65 typed 
characters. The computerized client had information on the etiology of several psychological 
problems, psychosocial history, developmental history, family background, and facts about the 
crisis. As the participant asked questions of the computerized client, it responded and waited for 
the participant to hit a keyboard key to continue. For example, the participant could ask, “I know 
this is difficult, but tell me what happened the night of the rape.” To this the computer would 
respond with the statement: 

Mark and I went out three times. Once to the movies, once to dinner, and finally on a 
camping trip with his friends. We shared a tent, and I told him I didn't want to make love 
with him, but he hit me and tore off my panties. He must have sensed that I was attracted 
to him. It's my fault. 

The computer program kept track of demographic information, participants’ self-ratings of 
accuracy, participants’ self-ratings of expertise, and a variety of variables involving the 
participant’s mode of inquiry. 
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The program contained 203 pre-set, paragraph answers that were created using responses 
from 22 participants during pilot testing. When these pilot participants asked questions the 
program could not answer, a new answer was created and feedback was sought as to perceived 
appropriateness of new client response. Also when the program inaccurately answered a question 
that was posed by one of these participants, the database was manually searched for the correct 
answer, then the program was changed to handle the participant’s linguistic style for future use. 

Potential answers were coded along a 3-point continuum: 0 points meant the answer 
contained general information, 1 point meant the answer was a general mental-health answer, and 
2 points meant the answer specifically involved the crisis at hand. These variables were coded by 
three, female, psychology interns who obtained an inter-rater reliability of .76 using Cronbach’s 
alpha. 

Dependent Measures 

The self-ratings of accuracy and expertise were obtained after participants finished asking 
questions of “the client.” Each of these two ratings was a single-item that employed a 7-point, 
Likert-type scale (where 1 = very low and 7 = very high) . The time taken to read the case (Time- 
to-Read-Case), ask a question (Time-for- Average-Question), and formulate a treatment plan 
(Time-for- Writing-Plan) were also calculated. 

The primary variables for the study were comprised of methods of categorizing the type 
and number of questions asked. These variables included: (a) the total number of questions asked 
(Total-Number-of-Questions), (b) the number of questions the computer program could not 
recognize (Unique-Questions), (c) the number of questions asked that were extensions of 



Expertise in Psychological Inquiry 

13 

previously asked questions (Follow-up-Questions), (d) the number of questions exploring material 
specifically connected to the case (Case-Related-Questions), (e) the number of questions 
regarding the client’s personal history (Client-History-Questions), the (f) number of questions 
exploring diagnostic criteria (Diagnostic-Questions), (h) the range of questions, (i) the number of 
questions the participant asked that were prompted by the client, (j) the number of questions 
asked involving suicidal ideation (Suicide-Questions), (k) homicidal ideation (Homicide- 
Questions), (1) drug or alcohol use (Alcohol-Questions), or (m) a combination of these items 
(Critical-Items). 

Procedures 

Participants were recruited through a variety of means from 139 sites in 24 states and six 
countries (Australia, Belgium, Canada, Netherlands, Norway, and the United States). First, 12 
internet postings for volunteers elicited 40 participants. Second, the Veterans Administration 
Hospitals in Loma Linda California and Anchorage Alaska provided 1 8 participants, including 2 
students, 3 interns, and 1 5 licensed psychologists. Third, a random selection of practicing 
psychologists from California was conducted using the Internet White pages (which lists the 
phone numbers of all known psychologists within a given area). The fourth and final group of 
participants (n = 60) came from randomly selected training centers in the Southern California 
area. 

Participants were randomly assigned by computer to one of three treatment groups: (a) 
clinical impression, (b) DSM diagnosis, or (c) treatment plan. The assignment to groups was 
performed with each new participant rather than by diskette. For example, if three people chose to 
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participate, they would each be randomly assigned to a group even though they used the same 
diskette to run the program. Once assigned to a group, the program would modify all the various 
instruction pages to reflect the assigned group. 

A total of 90 diskettes were returned. Some diskettes, though, contained data from 
multiple participants, which yielded the data for 138 participants. 

Results 

Descriptive Data Results 

The 25 main variables for this study varied from dichotomous variables to large, 
continuous time variables. 

Reliabilities were calculated for all main effect variables, excluding the time-related 
variables. These results are displayed in Table 1. 



insert Table 1 about here 



Two simultaneous, multiple-regression analyses were performed to examine the research 
questions involving years of experience. A Hotellings MANOVA and Discriminant Function 
Analysis with Scheffe post hoc was employed to examine the effects of training. 

Experience and Types of Questions Asked 

Experienced explained approximately 32% (R 2 = .32, F(5,132) = 12.69, p < .0001) of the 
variance for the types of questions asked. After employing a Bonferroni correction, four of the 
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five variables were significant related to experience. As psychologists gained experience, they 
were more likely to 

(a) attend to the client details (such as re-reading the case, asking questions prompted by 
the client, and asking questions directly related to the case description), F(5,135) = 

2.60, p=. 010 

(b) de-emphasize information gathering at either the beginning and end of the questioning, 
F(8,35)=2.66, g = .009 

(c) ask fewer questions, F(8,35) = -3. 16, g = .002, and 

(d) ask more diagnostic related questions, F(8,35) = 6. 12, g < .0001, 

Experience did not differentiate participants in the tendency to ask more unique or 

unpredictable questions, F(3,135) = 1 .02, g = .308. 

Academic training and mode of inquiry 

Participants’ level of academic training was divided into four categories including new 
students to psychology (n = 36), masters level therapists (n = 20), doctoral level students 
(n = 14), and doctoral level therapists (n = 68). A significant main effect of training was obtained, 
F(3,132) = 5. 19, g <001using the Pillai's Trace criterion. A total of 9 the 1 1 dependent variables 
achieved significance. 

A discriminant function analysis (DFA) was performed to assess the predictability of the 
significant differences created by level of training. The significant discriminant functions indicate 
that the variables entered into discriminant analysis identified differences between groups beyond 
what would be predicted by chance. The classification results, found in. Table 2 indicate that there 
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was a 69.12% correct prediction of group membership based on the predictor variables entered 
into the simultaneous discriminant analysis. The group representing students beginning their 
academic training was correctly predicted 73.7% of the time. The group of masters level 
therapists was correctly predicted 62.5% of the time. The group of doctoral level students was 
correctly predicted 64.3% of the time. The final group, doctoral therapists, was correctly 
predicted 69. 1% of the time. 



insert Table 2 about here 



Another interesting finding was the increase in prediction value once participants were 
grouped according to their assigned goal. Three additional DFAs were conducted to examine the 
effect of training on the types of questions asked when asked to perform specific tasks. 
Participants who were told they were asking questions in order to form a clinical impression were 
accurately predicted by level of training 91 .67% of the time. The group told they were asking 
questions in order to form a DSM-IV diagnosis were accurately predicted by level of training 
86.67% of the time. The final, group, who were told to form a treatment plan, were accurately 
predicted by level of training 82.50% of the time. 

Experience and Levels of Confidence 

Perceived expertise with date-rape clients and perceived confidence in judgment were 
significantly correlated (r = .532, p < .01), but neither variable was significantly correlated with 



experience. 
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Efficiency of Experienced Clinicians 

A simultaneous multiple regression was performed on experience as the independent 
variable and five dependent variables (Time-for-Writing-Plan, Time-to-Read-Case, and Time-for- 
Average-Question). The results were nonsignificant (R 2 = .12, F(2,135) = .92, p = .400). 
Experience did not appear to be related to efficiency of inquiry, at least as it was operationalized 
by these variables. 
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Discussion 

This study used a artificially intelligent computer program to explore how psychologists 
gather information about their clients. Evidence was found for the following: 

• The types of questions psychologists ask are largely explained by experience. 

• The methods psychologists use to explore client data can predict their level of 
academic training. 

• Experience is unrelated to case-specific, self-rated confidence or expertise 

• Experienced psychologists are not more efficient in gathering information than their 
less experienced colleagues. 

Su pported Research Themes 

Etringer, Hillerbrand, and Claibom (1995) reviewed the literature on expertise and 
concluded that experts use procedural knowledge, forward reasoning, and move toward 
automation. These studies frequently emphasized the cognitive process experts engaged in 
without examining the behaviors that followed. This study’s use of a mock clinical setting sheds 
additional light on the process of expertise by looking at the tasks on which experienced therapists 
choose to act. 

Ex perience and Information Gathering 

As psychologists gain experience, they being to ask questions differently. This 
corresponds to O’Byme and Goodyear’s (1993) findings that experts in psychology tended to 
have a “common clinical wisdom.” However, the “wisdom” of the therapists in this study, took a 
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somewhat different form than the wisdom depicted by O’Byme and Good year (1993). They 
found that experts tended to ask questions similar to those of other experts, while beginning 
therapists asked more unique questions. Experts were also more likely to ask questions about the 
client’s history, and they asked more questions overall. In this analysis, the “common wisdom” 
was identified for inquiries into diagnostic criteria, but most of the positions found by O’Byme 
and Goodyear were not supported in the Multiple Regression analysis. 

In this study, as the psychologists gained experience, they were not more likely to ask 
unique or unpredictable questions. Also in contrast to the O’Byrne (1993) study, experienced 
psychologists were not more likely to explore their client’s history or ask more questions (F(8,35) 
= -3 . 16, p = .002). Experience was negatively related to the number of questions asked, which 
supports the general theory that experience leads to a form of automation (Anderson, 1981; 
Dumont, 1991; Glaser, 1991). 

The “common clinical wisdom” supported by this study involved that was associated with 
increased experience involved a number was the tendency to attend to client details (as evidenced 
by re-reading the case, asking questions prompted by the client, and asking questions directly 
related to the case description, etc.), F(5,135) = 2.60, p = .010. Experience also assisted in a 
general de-emphasis of information gathering (e g., client history) and an emphasis on crisis 
intervention, F(8,35) = 2.66, p = .009. This tendency to favor intervention compares well with 
the tendency to explore diagnostic related questions, F(8,35) = 6. 12, p < .0001 . However, 
experienced participants explored these issues by asking fewer questions, F(8,35) = -3.16, p < 

. 002 . 
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It appears that experience helped therapists to “listen” to the client better. Nevertheless, 
experienced therapists had an agenda. Most of their crisis intervention questions came from 
seeking diagnostic information, such as “do you feel sad more days than not?” If asked, the 
fictitious client presented with symptoms of depression and anxiety. This fits the Dreyfus and 
Dreyfus (1986) model of developmental expertise. Experts depend more upon the immediate 
context of the problem than its origin. The uniqueness of the client involves how the present 
symptoms develop out of the client’s personal history. This may underlie the reasons why 
experience does not appear to play a role in asking history questions. Beginners may ask the same 
number of history and information questions as expects but they did not explore them in the 
context of the current symptomology. Experts appear to ask history questions to fill in the gaps 
created through their intervention questions. 

Experienced therapists primarily fill in the gaps of a developing theory of the client, which 
may explain the negative correlation between experience and the number of questions asked 
(t( 132) = -3.651, p = .0004). As Polkinghome (1992) argued, experts appear to use prototypes in 
their conceptualization of clients. They use forward reasoning to seek confirmatory or 
disconfirmatory of their developing hypothesis. 

One reason why the O ’Byrne (1993) study may have concluded that experience led to 
therapists asking additional questions was categorization of participants as experts. In this study, 
experienced was correlated with asking fewer questions but increased training was correlated with 
more questions. Another reason for the difference was O’Byme’s method of analyzing the data. 
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When dividing participants into beginners, intermediates, and experts and analyzed with a 
MANOVA, the results dramatically differed from the multiple regression. 

Although the MANOVA analysis was significant (F(2,135) = 5.071, p_< .0001), it 
suggested that experience was significantly, positively correlated with the number of questions 
asked (F(2,135)= 3.135, p = .046), and significantly, positively correlated with inquiries made into 
the client’s alcohol consumption (t(2,135)=15.35, p_< .0001). In the multiple regression analysis 
both variables were negatively correlated with experience (t(14,123)= -3 .335, p = .001, and 
(t(14,123)=-2.783, p = .0062). 

Though the reasons for the differences are not yet clear, the multiple regression analysis is 
believed to be a more appropriate statistical procedure in these cases, because of the data lost 
when converting continuous variables to categorical variables. Light, Singer, and Willett (1990, p. 
83) argue “on no account should perfectly good continuous predictors like these be artificially 
reduced to a small number of discrete categories.” 

Academic Training 

Level of academic training also had a significant effect (F(3,132) = 5.19, p < 001) on 
clinicians’ modes of inquiry. As levels of academic training increased, so did experience (r = .646, 
P < .01), but training still has a significant effect even when a MANCOVA was employed to 
control for the effects of experience (F( 1,131) = 1.91, p = .044). There is no apparent problem 
with multicollinearity (Tabachnick & Fidell, 1983). Experience and training measure different 
constructs. 
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Higher levels of training were unrelated 8 of the 1 1 variables explored. Two of the three 
significant variables contradicted the findings for experienced participants (total number of 
questions (F(2,135) = 4.02, p = .020) and client history questions (F(2,135) = 5.89, p = .004). 

The only variable corresponding to findings for increased experience was the tendency to follow 
up on previously asked questions (F(2,135) = 6.38, p = .002). 

The post hoc analysis did not reveal any significant differences between doctoral students 
and psychologists with doctoral degrees, but the predictability of each group within the functions 
of group centroids was highly significant. Participants were correctly classified into their 
corresponding groups over 69% of the time, and even more accurately when participants were 
divided into specific tasks. 

This ability to predict participants’ level of training did not follow a linear progression. 
There were two functions found in the discriminant functioning analysis. Both functions played 
key roles in discerning between experts and novices. The variables loading on function 1 were 
Diagnostic-Questions, Follow-up-Questions, Critical-Issue-Questions, Preference for Intervention 
over Information, and Total Number of Questions. All of these variables involve either the 
process of investigating specific client symptoms or maintaining a broad approach to the issues 
explored. This function will be referred to as diagnostic inquiry. 

Function 2 consisted of Case-Related-Questions, History-Questions, and Unique- 
Questions. These variables involve the process of attending to client cues or exploring non- 
diagnostic, client issues. This factor will be referred to as detail inquiry. 
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An apparent curvilinear relationship exists between the academic training and the two 
functions. Participants in the least trained group scored lower in diagnostic inquiry any other 
group, and lower than the masters and early doctoral groups in detail inquiry. The next two 
training groups, predictably, scored higher on both functions. The surprising component was the 
performance of the doctoral-level psychologists. They scored lower than all other groups on the 
detail inquiry category, but scored higher than all other groups on the diagnostic inquiry function. 
Rather than follow the client’s prompts or emphasize history issues, the psychologists focused on 
the problems at hand such as suicide, depression, and alcohol. This finding corresponds to Patel 
and Groen’s (1991) study concerning intermediate or mid-term students. In their study, 
intermediates recalled more irrelevant and more relevant information than did experts. Similarly, 
in this study, the group with the most training appears to have underemphasized issues brought 
out by the client in order to favor the most significant areas of inquiry. 

The tendency for the accuracy of prediction to dramatically increase when participants 
were examined by randomly assigned treatment groups, also corresponds to the literature on 
expertise and automation. It provides the increase structure that Hillerbrand and Claibom (1990) 
found essential in the process of automation. 

Self-Rated Confidence 



ERIC 



24 



Expertise in Psychological Inquiry 

24 

There is a long history of literature indicating that therapists will report higher levels of 
confidence than they merit (Dumont, 1993; Dunning, Griffin, Milojkovic, & Ross, 1990; 

Glidewell & Livert, 1992; and Tracy, 1991). However, according to the literature, psychologists 
tend to become more confident with experience (Arnold, 1993). 

In this study, neither self-rated confidence or self-rated expertise were significantly 
correlated with any of the main variables. Confidence did not increase with experience, nor did it 
positively or negatively affect the participants’ investigation. 

Efficient Therapists 

The literature on expertise in most examined fields strongly supports a connection 
between efficiency and experience (e.g., Koupemik, 1995). Once certain procedures become 
automated, it is reasonable to assume that they would take less time to formulate their mode of 
inquiry. In this study, no such connection was found. 

O ’Byrne (1993) found a significant difference between the time spent by experts and 
novices in asking questions (M = 8.71 minutes for novices; M = 10.00 minutes for experts). In 
this study, the average time spent asking questions was considerably longer (M = 29.30 minutes 
for all participants with a standard deviation of 16.47 minutes). Participants ranged in the time 
spent asking questions from a low of 1 1 minutes to a high of 1 1 7 minutes. The computer probably 
contributed to the time increase. There was no human interaction during the data collection, 
which provided participants with more perceived autonomy. Nevertheless, the multiple regression 
analysis found no significant effect for experience on the time spent asking questions (F(5,132) = 
.726, p = .604). Experience also had no statistical impact on the time spent formulating diagnosis 
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questions, but it did assist in reducing the time spent formulating questions about the client’s 
history (r = -.160, p < .001). 

The general lack of efficiency may be explained by the complexity of the task at hand. For 
example, expert writers do not automatically applying in their discipline. The art involves a 
constant interaction between knowledge, experience, and the task at hand. This often makes 
writing more difficult for experts than for novices; it becomes a more complex process. Novice 
writers simply write out their thoughts, but expert writers plan, revise, and mine their knowledge 
base for the best possible method of communicating a thought (Flower, 1979). 

Summary and Conclusions 

This paper is significant in that it provides evidence of the effects training and experience 
have on the types of information psychologists may ask of their clients. In also provides a 
window into the ways psychologists might automate their modes of inquiry and conceptualize 
cases. 

The results for this study, however, should be viewed with some caution for several 
reasons. First, the moderate sample size was not large enough to overcome the differences 
between the participant pools. The various samples differed on several keys areas. Three of the 
four groups were selected randomly from specific settings, but the fourth group was comprised of 
volunteers. This highly trained, computer literate group may have effected the variance of the 
main variables. 

Second, the medium of the computer may have changed the ways the participating 
psychologists may have handled the case presentation. Part of this study investigated the way 
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atypical problems changed psychologists’ modes of inquiry. The moderating effect of the 
computer may have made the entire process “atypical.” Even if the types of questions asked were 
similar to those the participants would have asked a client, it is likely that the time spent asking 
questions differed in this study. Participants may have enjoyed the process of taking as much time 
as they wished without being prompted to continue. This freedom likely impacted the therapists’ 
inquiries. 

The automated client responses and lack of sensory input also may have changed the 
questions that were asked. Although the computer program was very successful in answering 
questions, true dialogue was not possible. The therapists were also unable to react to client cues. 
The inability for the computer to produce tones, gestures, pauses, or other nonverbal cues may 
have changed the therapists’ mode of inquiry. 

Third, there is no way of knowing if participants would have reacted differently if the case 
were about a male client. Gender effects should be explored in future studies. Nevertheless, the 
results of this analysis provide an important first step to understanding the ways experience, 
academic training, cognitive complexity, and assessment task change psychologists’ modes of 
inquiry. 

Despite these weaknesses, the study appears valid and provides increased depth and 
perspective to the O’Byme (1993) study. The weaknesses discussed here do not appear 
significant enough to overcome the strong effective sizes found in this study. 

It should be noted that the findings here suggest that experience and training play a 
significant role in the investigative process psychologists employ when assessing a client. This 
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study cannot posit, however, the superiority of one method over another. It is possible that the 
methods used by beginning psychologists are functionally equivalent to that that of advanced 
psychologists. Further studies should explore the relationship of the assessment methods outlined 
in this study with diagnostic accuracy or increased therapeutic outcomes. Previous studies 
comparing beginning and advanced psychologists found little differences between the diagnostic 
outcomes of the two groups. Now that an effective tool exists to predict expert vs. novice group 
membership based on the mode of inquiry used, there is reason to believe that the two groups 
should arrive at different conclusions. 
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Table 1 

Reliability of Main Effect. Excluding Time Variables (N = 138) 
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Table 2 

Summary of Simultaneous Regression Analysis for Variables Predicting Effects of Academic Training on 
Mode of Inquiry fN = 136) 
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Percent of "grouped" cases correctly classified: 69.12% 

* indicates significance greater than chance, with chance being equal to 25%. 
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